Skip to content

ARROW-13087: [R] Expose Parquet ArrowReaderProperties::coerce_int96_timestamp_unit_#12070

Closed
paleolimbot wants to merge 3 commits into
apache:masterfrom
paleolimbot:r-int96
Closed

ARROW-13087: [R] Expose Parquet ArrowReaderProperties::coerce_int96_timestamp_unit_#12070
paleolimbot wants to merge 3 commits into
apache:masterfrom
paleolimbot:r-int96

Conversation

@paleolimbot

Copy link
Copy Markdown
Member

This PR adds a setter and getter for ArrowReaderProperties::coerce_int96_timestamp_unit_.

Reprex:

library(arrow, warn.conflicts = FALSE)

tf <- tempfile()
write_parquet(
  data.frame(a = as.POSIXct("2001-01-01 12:34:56.789", tz = "UTC")),
  tf,
  use_deprecated_int96_timestamps = TRUE, version = "1.0"
)

props <- ParquetArrowReaderProperties$create()
props$set_coerce_int96_timestamp_unit(TimeUnit$MILLI)
props$coerce_int96_timestamp_unit() == TimeUnit$MILLI
#> [1] TRUE

(p <- read_parquet(tf, props = props, as_data_frame = FALSE))
#> Table
#> 1 rows x 1 columns
#> $a <timestamp[ms]>
#> 
#> See $metadata for additional Schema metadata
p$a
#> ChunkedArray
#> [
#>   [
#>     2001-01-01 12:34:56.789
#>   ]
#> ]

@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@kszucs kszucs requested review from jonkeane and thisisnic January 14, 2022 13:05

@thisisnic thisisnic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looks good!

@kszucs

kszucs commented Jan 18, 2022

Copy link
Copy Markdown
Member

Thanks @paleolimbot and @thisisnic!

@kszucs kszucs closed this in 94d4719 Jan 18, 2022
@ursabot

ursabot commented Jan 18, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 17bf54a and contender = 94d4719. 94d4719 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.0% ⬆️2.14%] ursa-i9-9960x
[Finished ⬇️0.17% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants